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ABSTRACT 
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multi-state network of school districts and other educational 
agencies that actively promotes science improvement through the 
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WHAT IS THE SCIENCE CURRICULUM AND 
ASSESSMENT PROJECT? 

The project is a mulli-stale network of school districts and other education agencies that actively promotes 
science program improvement through the alignment of curriculum, instruction and assessment It was formed 
in 1982 by members of the Northwest Evaluation Association in response to the obvious need for better 
assessment tools in science. 

The project is probably best known for its quality workshops and its science test item collection. An annual 
Winter Science Institute provides staff development training to administrators and teachers in daiabased 
program improvement, curriculum development, instructional strategies related to science concepts and 
processes, program evaluation and assessment procedures, and the management of change. The science test 
item collection currently contains over 8,000 items covering a wide range of science topics as well as major 
concepts and processes that cross science disciplines. (Sec appendix A for an inventory of items by goal 
category). Each year, new items are written for selected grade levels and goal areas to broaden the coverage of 
the test items. TTie project's long-range goal relatf:d to the collection is to provide districts with a resource that 
will enable them to adequately assess all science areas in all grade levels. 

Support for project activities comes primarily from an annual membership fee paid by participaang agencies. 
Steering committee members include classroom teachers, school district curriculum and evaluation specialists, 
principals, university professors, and specialists from the Oregon Department of Education and Northwest 
Regional Educational Laboratory. 

WHAT ARE THE PROJECT'S 
MAJOR AREAS OF WORK? 

Each year the project develops a schedule of work designed to support local program improvement efforts. Over 
the past five years, ongoing work has included the development of materials, procedures and staff development 
activities in the following areas: 

• Improving science programs through the alignment of curriculum, instruction and 
assessment. This includes materials and procedures for helping school districts: 

-Design a K-12 science program 
-Evaluate and select instructional materials 
"Identify goals to be assessed and develop evaluation designs 
-Develq) curriculum-aligned assessment instruments useful for 
classroom and program evaluation purposes 

• Developing science test items for purposes of program and student 
assessment in the following areas: 

-Science Concepts and Processes 

"General Science (K-10) 

"Biology 

"Chemistry 

"Physics 

"Earth/Space Sciences 

The science test item collection will enable disuicts to develop a valid and reliable testing program, aligned to 
local cumculum. The current collection contains items covering grades K-12. All items are referenced to a 
comprehensive science taxonomy. ; 
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Within the next ten months, the collection will be divided into six separate item banks. Through extensive field 
testing over the next several years, all of the test items will be calibrated (i.e.. a difficulty level for each item 
will be determined in relation to all other items in the bank. The electronic version of the te.st items 
will be available in the fall of 1988. 

• Increasing teacher understanding of the key :oncepts and processes thai cross science disdplines. 
This includes assistance with clarifying concepts and processes, developing instructional objectives 
desirable or concept and process learning, selecting appropriate test items, and developing 
instructional methodologies, learning activities and materials. 

• Providing staff development activities to increase the skills of teachers and administrators in the 
areas of data-based prograr. improvement, curriculum review and planning, assessment, and the 
management of change. 

• Promoting and supporting the development of cooperative working agreements among .school 
districts, colleges and universities, regional educational laboratories, and .state departments of 
education. 

WHAT ARE THE SPECIFICATIONS 
AND REVIEW PROCEDURES FOR TEST ITEMS 
IN THE ITEM COLLECTION? 

Introduction tn the rollprtinn 

For the past five years, the project hac placed a strong emphasis on iteir. writing activities in order to create a 
pool of well-constructed test items that assess student learning of science subject-matter goals and science 
concepts and processes. 

The current item collection conU-^ns 8.000 test items in card stock form. Approximately 2.000 new items are 
added to the collection each year. An estimated 16.000 items will be needed to adequately assess science subject- 
matter goals, concepts and processes in all courses and grade levels. 

The majority of the items currently in the collection were written by teachers who receive ongoing instruction 
in the techniques of writing high-quality multiple-choice test items. The remainder of the items are public 
domain items which have been revised to conform to NWEA specifications. Regardless of the source, the items 
currently in the collection arc judged to be ready for field tesung. 

Items that survive field testing will comprise the intial NWEA Science Test Item Collection. Through 
.subsequent field testing, the calibration (difficulty level) of each item will be determined using iicm-rcsponse 
theory (IRA) methods. Initial research field testing indicates that all of the items cannot be calibrated using a 
single, underlying scale. The following six scales have been identified to date: 

• Concept and Process Items 

• General Science (K-8) 

• Biology 

• Chemistry 

• Physics 

• Earth/Space Sciences 

The items are being divided into separate banks corresponding to the six scales listed above. Once this 
transition has been completed, the banks will be available only in electronic form. The card stock versions will 
no longer be produced. It is anticipated that the electronic form of the six collections will be available in the 
fall of 1988. 
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The Tfpm rnllprtinn 



The purpose of this section is to describe the criteria and specifications used by the Science Curriculum and 
Assessment Project in the development of the item collection. One of the major innovations of the project is 
the inclusion of items for both (1) conventional subject-matter topics and (2) the concepts and processes of 
science. 

Science Topics 

The item collection copta* is items related to the following science topics:(*) 

A. Life Sciences 

• Tlie Cell 

• Zoology 

• Botany 

• Simple Life Forms/Diseases/Funclions 

• Ecology 

B . Earth/Space Sc iencc 

• Geology 

• Oceanography 

• Meteorology 

• Space and Astronomy 

C. Physical Sciences 

• Measurement 

• Matter: Propenies/Slates/Changes/Struciure 

• Periodic Law/Table 

• Acids, Bases, and Salts 

• Giemical Kinetics/Equilibrium 

• Oxidation-Reduction 

• Electrochemistry 

• Organic Chemistry 

• Nuclear Chemistry 

• Photochemistry/Speclroscopy 

• Chemistry and Society 

• Biochemistry 

• Motion 

• Gravity 

• Electricity and Magnetism 

• Energy/Transformations/Encrgy Resources 

• Light 

• Modem Physics 

D. History of Science 

E. Careers in Science 

F. Laboratory 

G. Vocabulary 



(♦) The NWEA Science Item Collection Index provides a complete listing of the subcategories contained within 
each major topic. 
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Science Concents; and ProcewK 



In addiuon lo the convcnuonal lopi^ al caicgoncs of science instruclion, ihc piojccl fiKuscs on maior concepts 
ami processes of science learning. Wilh the inclusion of concept and process iiems in ihe colicclion it is 
possible to design sophisticated science achievement tests that assess both subject-matter knowledge and higher- 
level thinking sknls. Given that science knowledge conunues to accumulate and change at a rapid rate the 
objccuves of teaching conceptual and m-thodological understanding of science will require matching assessment 
methods. The item collccuon's concept ?Jid process items make possible the design of such tests. 

The collection contains items related to the following science concepts and processes: 



H. Concepts 



Cause-Effect 

Change 

Cycle 

Energy-Matter 

Entropy 

Equilibrium 

Evolution 

Field 

Force 

Fundamental Enuiies 

Gradient 

Interaction 

Invariance 

Model 

Order 

Organism 

Perception 

Piobabilily 

Population 

Quantification 

Replication 

Resonance 

Scale 

Significance 

Symmetry 

System 

Theory 

Time-Space 

Validation 



I. Processes 

• Classifying 

• Communicating 

• Controlling Variables 

• Defining Operationally 

• Designing Experiments 

• Formulating Models 

• Hypothesizing 

• Inferring 
Interpreting Data 

• Measuring 

• Observing 

• Predicung 

• Questioning 

• Using Numbers 

• Relating Time-Space 



A brief definition of each concept and process is provided in Appendix B. 



Item SDecificafion^ 

To develop a high quality test item bank, the project relies on the cooperation of many educators. Curriculum 
experts and teachers periodically review the items in the collection. Through review, careful editing and field 
testing, flaws and ambiguities are removed from items that enter the collection. The project's recommended 
procedures for reviewing items will be described subsequently. The procedures are based on actual experience 
with the challenges and rewards of reviewing thousands of lest items. 
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The project's item development and review process is shown in Figure 1. Here are some points to note in 
looKing at the flowchart 

• Items enter the item collection from three main sources: 

1 . Annual writing workshops conducted by the project, 

2. Local district production of new items, and 

3. Other item collections or catalogs obtained from educational agencies and research centers. 

• Items receive peer review either at the workshops on item writing or from the project editorial 
board 

• Flawed items may be identified by workshop participants, users of the collection, the editorial 
board, or on the basis of field test statistical results. 

Item Development an d Review Prnrp^sis 



Item Wriung 
Workshops 



• Wnie Item 

• Peer revtew 

• Staff edit 



YES 



YES 



liem 
Collection 



YES 



T 



Local 
Production 



Other 
Authoritative 
Sources 





Project/Local Item 
Screening Workshops 






• Review items 

" Add 5th response 

• Staff edit 


NO 

► 


^ — 


YES 







YESmEVISED 



NO/FLAWED ITEMS 



(teacher/district selection) 



Loca distncts 
field test 



NO/FLAWED ITEMS 



(teachers giving tests) 
NO-STATISTICALLY FLAGGED (PBS < 3) 



MAYBE 



Editorial 
Board 



RECOMMEND MORE WORK 



NO/DELETE ITEM 
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Steps in the Item Development and Review Process 



Tlic first step in adding itc.Tis to ihc collection is ihc dclcrminaiion by project staff or local project members that 
(I) items must be written to fill identified gaps, or (2) existing items can be modified from authoritative sources 
(e.g., the National Assessment of Educational Progress). The process of item generation and review for each of 
these two options can be summarized as follows: 

L New Items: Step?; II. Existing Items: Steps 

1 . Item writing workshops I . Item screening workshops 
or local production or local screening 

2. Apply criteria for item writing 2. Add 5th response if needed 

All Items 

3. Peer review at workshops 
or in local settings 

4. Code items to the appropriate 
goal categories 

5. Assign a grade level range 
(K-3.4-6. 7-9. 10-12) to 
each item 

6. Apply criu^ria for elimination 
of racial or gender bias 

7. Enter acceptable items into 
thecolle ion 

8. Flawed items are sent to the 
editorial board and revised or discarded 

9. Field testing 

10. Statistically flagged items 
are sent to editorial board 



Writing Npw Items 

Each year the project conducts item writing sessions for project members and other interested educators. These 
sessions arc designed to: (1) introduce participants to the writing of concept and process items, and (2) train 
participants in the principles of writing good multiple-choice test items. Two major principles of good item 
writing are used as guideVnes: 

1 . To keep each item free from "test-wisencss" factors that allow students to derive the answer by 
guessing or test-taking skills unrelated to science achievement. Examples are implausible wrong 
answers, cues or hints to the answer, the correct answer being the longest alternative in a multiple- 
choice format, etc. 

2. To make sure that the item is "science-achievement pure." By "science achievement" we mean the 
facts, concepts, principles, ^nd skills that arc the intent of the science curriculum being assessed. 
By "pure" we mean that the i!em does not simultaneously measure knowledge in science and extra 
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skiUs such as reading abilily (as in ihe case of items ihal are too wordy), logical-lhinking (where 
the answer is derivable by an aitenUve. logical person who may know nothing of ihe science 
objective being tested), general intelligence, or ethnic/culluial knowlcge. 

In other words good items arc those that direcUy and exclusively test the targeted science fact, concept 
principle, knowledge, skill, or sensiUvity without the interference of guessin^ test-wisencss. reading ability, or 
other skills or factors unrelated to science achievement 

There are a number of lists of "item wriUng rules" in the literature on educaUonal tesUng. Below is a suggested 
list for implementing the two major principles of item writing. 

Criteria for NWFA M„lffnli>,rhnirP it^^^ 

1. Freedom firom racial, ethnic or sex-role bias or stereotyping. (Sec Appendix C: Criteria.) 

2. Clear, j^prq^riatc. simplest possible text, with terms used accurately. 

3. One righi or best answer, with plausible dislractors. If the "best" answer is requested, criteria for 
choice of "best" is given. 

4. No give-aways: answer choices about the same length, no grammatical cues. ete. 

5. All information needed to choose the correct answer is given except that for which the item tests. 
In other words, students who have command of that process or concept or information tested can 
get the item correct, whether or not they have rented background information that is commonly 

6. Language used in the item is standard usage, spelled and punctuated correctly and appropriate to 
grade level and subject 

7. Graphs or illustrations are clear and accurate. 

8. Item is logically content valid. That is. it measures tf goal to which it is related. (If students get 
the item nght. they probably do have the knowledge or skill described by the goal' if tliey ger the 
Item wrong, they probably don't) 

9. Correct answer is keyed. 

10. All items have five alternatives (correct answer plus four dislractors). 

In addition to these criteria, workshop participants arc given a set of recommended steps in item writing (Sec 
Appendix D; Writing Good Test Items - How?). 

Screening Fiyislim,lt£im 

The project screens items obtained from existing sources such as ot**^r item banks or catalogs. Workshops and 
a traming manual are used to prepare teachers to review and screen promising items idenUfied by project staff 
Screcners use the Criteria for NWEA MulUple-Choice Items to judge the suitability of the items. Appendix E 
provides a list of additional characteristics examined when item screening takes place. Also, to provide 
uniformity in the item collccUon. and to reduce any ptotcnUal guessing factors, the number of item altemaUves 
(correct answer plus dislractors) is set at five. If existing items only have four alternatives, a fifth is added by 
the screcmng team. Each item is coded to one goal category and a grade level estimate is given using the levels 
f'\7'?* ^^^2. Item screeners are selected if they have both knowledge of the topical area and 

familiarity with leaching at the grade level of the items to be screened. On the initial pac-. all itrms arc 
screened by at least two people. 
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Functions of the Kdilnrial ni^f^^d 

The project's editorial board meets on a monthly basis to review items that have posed difficulites for users of 
the collection or field test participants, or items that have been flagged in field testing. The role of the board is 
to attempt a judicious review and to resolve ambiguities, not to do extensive revisions of items. The board is 
guided by the following options for routing reviewed items: 

1 . Revise the item and send it back into the collection. 

2. Reject any items that either will take too much time to revise or arc unrecoverable. 

The board is further guided by the principle that it takes a lot more time to repair an item than to find a new 
(good) item. 

Field Testing of Ttemfi 

All items in the collection will eventually be field tested on samples of 200 or more students and caMbrated 
using item response theory or Rasch model methodologies. In addition to this long range goal by the project, 
local districts and project members are encouraged to conduct conventional item analyses on any tests 
constructed from the item collection. Where samples of 150 or more students are included in field testing, the 
project requests a computer tape or diskette containing the item response data. In addition, all field testing uses 
of items in the collection are encouraged to include computer analyses of item characteristics such as 
discriminaiion (correlation between the item and the total test score). A criterion of correlations lower than .30 
has been set by the project to help identify potentially flawed items. 

In addition to statistical analysis of items, field testers are encouraged to send project staff any comments from 
teachers or students that may have been noted during a field test iryout of items Any potential ambiguities, 
typographical errors, or other flaws may be indentified in this way. 

Additional Therks 

To maintain the quality of the item collection and to direct future searches, two record forms have been 
developed: The Flawed Item Form and the Content Reaction Form. 

Flflwgd Ugm Form> During the process of reviewing items, or after administering a test, items may be 
discovered that are flawed in some, way (e.g., incorrect key, inco*Tect goal information, or incorrect 
information in the item). These flaws are noted on a Flawed Item Form. A separate form is completed 
for each flawed item. These forms are then reviewed by the editorial board. 

Completion of the Flawed Item Form is an important step in Uie de vetopmenl oi the NWEA Science 
Test Item Collection. It is one more way to detect flawed or weak items in addition to liiose detected 
by the item review panel and during the field testing process. 

Content Reaction Form. This form is used in the following situations: 

• To record additional goal categones for which no items cunrenlly exist in the item collection, and 

• To record goal categories for which an insufficient number of items currenUy exists m the iten 
collection. 

Feedback received on the Content Reaction Form helps direct future item searches and item 
development since it identifies gaps that currently exist in the item collection. 
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WHAT PRODUCTS ARE AVAILABLE 
THROUGH THE PROJECT? 

Ptojwi members receive dl of the materials listed below and any new products developed during iheir year of 
membership. All materials have been field tested and used by participating districts. 



CurriPHliim Surveys 

Six science curriculum surveys that determine the emphasis teachers place on a wide range of topics, concents, 
and processes are available for o t- f 

• Grades K-6 

• Middle School/Junior High 

• Biolo^ 

• Chemistry 

• Physics 

• Earth/Space Sciences 

Purchase of the survey masters includes written permission to reproduce the copyrighted materials for use within 
an individual school districL Non-member price; $20 per survey; $100 per set. 



Survey Scoring '^nd Renortinp gprvj^i>^^ 

The project^s scoring service provides rapid turnaround of reports for each grade level and each course surveyed. 
Special reports are also available (fees to be negotiated based on the type of report requested). 

Curriculum Plunninp MafoWaU 

A Curriculum Planning Handbook and support materials (a complete set of seventeen planning matrices) arc 
available to help curriculum committees answer questions such as: 

• How do we develop a scope and sequence that will align the curriculum from K- 1 2 and be used and 
valued by teachers? 

• Which textbooks and other instructional materials best meet our local needs? 

• Which portions of the curriculum warrant attention when developing and/or selecting tests and 
other assessment insuiiments? 

• How do we incorporate major concepts and processes into exisUng science programs? 
Non-member price: $150. 



The project has developed a collecUon of 8,000 test items printed on card slock, from wl ich teachers and 
cvaluauon specialists can develop high quality tests. The collecUon covers a wide range of science topics as 
vm as inajor concepts and processes that cross science disciplines. Most of the items in the collecUon have 
been written by skilled item writers trained and supervised by Uie project. All items are referenced to a 
comprehensive science taxonomy. 
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Ongoing field testing of the items identifies potentially flawed items. These items are subscqucnay reviewed by 
a panel of teachers and aie either revised or deleted from the collection. 

New test items arc added to the collection each year. These annual updates are available thmugh continuing 
membership in the project. 



Electronic Versinn of f^^y Sfiffl^f T^t Ttrmi 

The electronic version of the science test iterns will be available during the 1988-89 school year for the first 
lime. Support materials and training sessions arc currently being developed to provide assistance in the use of 
the new system. 

SpccificaUons for the computer equipment and software needed for the electronic version are available upon 
request Some software will be available for purchase Uirough the project; some through Assessment Systems 
in Minnesota. 



Other PoMirafinnts 

A Focus on Science Concepts by David C. Cox. This series of concept papers is designed to assist 
teachers in implementing concept-based science programs. Each paper focuses on a specific science concept and 
includes a definition of the concept in teacher language using examples from the contemporary curriculum; 
instructional objectives desirable for concept learning; sample test questions with a commentary as to their 
aH)ropriateness for measuring concept learning; and a list of references for additional reading about the concept 
Oyer the next year, papers will be prepared covering 29 key science concepts. The first five papers in this series 
will be blished this spring covering Cycle, Equillibrium, Force, Model and Perception. 

High School Science Programs: Managing for Excellence by the Northwest Regional Educational 
Lab A systematic and collegia! approach to managing building-level science programs, including a process for 
aligning the curriculum; five steps to higher student achievement; and the role of the principal in supporting 
program improvement This publication combines findings from the study of effective schools and teachers 
with a goal-based approach to program management. 

The Study of Learning Environments edited by Barry J. Fraser. Eight papers presented at the 1985 
Annual Meeting of AERA. including "Two Decades of Research on Perceptions of Classroom Environment and 
"Nature and Role of Target Students in Science Classroom Environments." 



WHAT ARE THE BENEFITS OF MEMBERSHIP 

Members receive all materials developed by the project over the past five years as well as new materials 
developed during their year of membership. In addition, they receive a 15% discount on workshop 
registration fee<:. 

Members becL,.ie part of a growing multi-state team dedicated to developing quality materials and 
practical staff development activities which provide members with the skills and procedures necessary 
to affect cha^ at the local level. 

Membership offers the unique opportunity to make a concrete, positive contribution to the imp/ovement of 
science education. Many members serve on the Steering Committee, the project's policy and program decision- 
making group. Active participation in workshop planning, item writing, field testing and other project work 
also increases the skills of individual members. 



WHAT ARE THE ANNUAL MEMBERSHIP FEES?* 



NEW MEMBERS $2,000 
RENEWALS $1,000 



Fees for SPONSORING AGENCIES 

Membership agreements with sponsoring agencies that join on behalf of a group of school districts are negoUated 
on an individual basis with the Project Director, Sponsoring agencies might include a consortium oflocal school 
districts, an education service district, or a state department of education. 

♦The membeiship year runs from July 1 though June 30. Fees include membership in the Northwest Eviluition Associition. 



HOW CAN MY AGENCY 
BECOME A PROJECT MEMBER? 

Simply return the membership form. Use the order form to indicate which materials you wish to receive. 
Enclose a check (payable to NWEA Science Project) or purchase order. Please allow thrcec to four wee^s for 
shipment of the Science Test Item CoIIecUon. Shipping charges for the collection will be added to your 
invoice. If you wish to receive the electronic version of the Science Test Items, please call Susan Smoyer 
(503/378-4157) for information about compter hardware and software requirements, costs, and vendors. 

ARE MATERIALS AVAILABLE 
FOR PURCHASE BY NON-MEMBERS? 



Yes, use the order form to indicate which materials you wish to purchase. Enclose a check (payable to NWEA 
Science Project) or purchase order. Please allow three to four weeks for shipment of the Science Test Item 
Collection. Shipping charges for the collection will be added to your invoice. If you wish to order the 
electronic version of the Science Test Items, please call Susan Smoyer (503/378-4157) for more information 
about computer hardware and software requirements, costs» and vendors. 
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NWEA SCIENCE TEST ITEM COLLECTION 
INVENTORY OF ITEMS/GOAL/GRADE LEVEL 
February 1988 



GOAL CATEGORY 


Number of Items 




K-3 


4-6 


7-9 


10-12 












The Cell 




29 


56 


106 


Zoology 


87 


174 


229 


291 


Botany 


74 


73 


118 


127 


^imnlp T iFp T7nrmc 

•jiiiipic irfiic mriub 


2 


8 


35 


53 




17 


70 


177 


81 


FARTH/^PAPF ^PTPNin^^ 










Opftl ft CT V 


97 


138 


247 


30 


11 pjkom ncTrQ nnv 

L/ccdiju^idpiiy 


4 


24 


190 


73 


ivic icui uiuj^ y 


44 


79 


116 


7 


opdcc anu /\siri/iiumy 


44 


80 


156 


18 


rrl I olLAL oClbNCbo 










Measurement 


10 


7 


42 


67 


Matteri Properties 


16 


22 


57 


30 


Matter: otates oi 


30 


22 


23 


32 


Matter! Changes 


16 


40 


81 


125 


Matteri Structure 


2 


38 


139 


303 


Periodic Law/Table 






23 


23 


/\tiasy DaseSy ano oaiis 






22 


31 


Chemical Kinetics 






4 


56 


i^nemicai cquiiiDrium 


_ 






27 


wxiQaiion 


- 


- 


2 


22 


Electrochemistry 






3 


2j 


Organic Chemistry 


- 


- 


3 


20 


Nuclear Chemistry 






23 


83 


Photochemistry 








1 


Spectroscopy 








1 


Chemistry and Society 






1 


2 


Biochemistry 






4 


16 


Motion 


13 


25 


74 


369 


Gravity 


4 


9 


22 


17 


Electricity and Magnetism 


12 


42 


114 


238 


Energy 


79 


79 


159 


147 


Energy Resources 


4 


50 


47 


7 


Light 


9 


20 


68 


139 


Modern Physics 








3 


HISTORY OF SCIENCE 




1 


9 


3 


LABORATORY 


1 


2 


19 


9 



ERIC 



15 
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GOAL CATEGORY 


Number of Items 




K-3 


4-6 


7-9 


10-12 


VOCABULARY 


2 


2 


10 


a 


CONCHKl^j 










Cause-Effect 


I 


J 


7 


1 1 


Change 




3 


10 


3 


Cycle 


L 


4 


13 


0 


Energy-Matter 


A 
«♦ 




10 


n 

y 


Entropy 






3 


1 o 

18 


Equilibrium 


1 
1 




1 J 




Evolution 


'1 
J 


A 


1 n 
1 u 


1 9 
1 Z 


Field 


1 
1 


1 
1 


7 




Force 




1 1 
1 1 


7 


A 


Fundamental Entities 


10 

1 \J 


1 1 
1 1 


1 

1 J 


Q 
0 


Gradient 


1Q 


A 
t 


^z 


1 0 
1 u 


Interaction 




A 

\j 


A 


J 


In variance 




A 
t 




Q 


Model 


1 
1 


9 


1 9 
1 Z 


14 


Order 


Q 


1 A 
1 0 


1 7 
1 / 




Organism 


1 
1 


in 

1 u 


7 




Perception 


1 u 


0 


1 0 


Q 
O 


Probability 




0 


9^ 
Zj 


91 
Zj 


Population 


J 


1 1 
1 J 


1 0 


1 Z 


Quantification 




0 


1 A 
lU 


c 

J 


Replication 




2 


4 


8 


Resonance 




1 


1 


9 


Scale 


1 


8 


23 


23 


Significance 






z 


o 
0 


Symmetry 


J 


1 0 


91 
Zl 


1 o 


System 


9 


1 J 


14 


Q 
O 


Theory 






Q 


94 
Z4 


Time-Space 


3 


12 


10 

1 \J 


1 

1 J 


Validation 




1 
1 


9 
z 


7 


PRC 










ci, :;v;. . 






14 


3 


Ce^i'n iv.juing 


Q 


1 A 

14 


o 

0 


2 


T'o :Lix..»ing Variables 


1 


1 A 

14 


OC 

z^ 


7 


"'^^ iiiing Operationally 


0 


1 Z 


lU 


1 


ciesigning Experiments 


/I 


1 Z 


14 


9 


Formulating Models 




z 


o 
0 


5 


Hypothesizing 


4 


2 


15 


5 


Inferring 


JO 






1 A 
1 4 


Interpreting Data 


15 


52 


79 


99 


Measuring 


8 


33 


20 


6 


Observing 


8 


25 


4 


6 


Predicting 


31 


15 


23 


8 


Questioning 


3 


2 






Using Numbers 


2 


10 


11 


7 


Relating Time-Space 


5 


6 


6 


1 



lb 

ERIC 



15 



GOAL CATEGORY 


Number of Items 
K-3 4-6 7-9 10-12 


SKILLS 


3 2 3 1 


INTERESTS 


1 


VALUES 


1 4 2 


INTERACTIONS 


1 


CHARACTERISTICS 


3 1 3 
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APPENDIX B 



SCIENCE CONCEPTS 

CAUSE/EFFECT (A relationship of events that leads 
one to believe that "nature is not capricious." 
Enables predictions to be made.) 

CHANGE (The process of becoming different in which 
the rate of change may vary from very fast to very 
slow so that it may be unnoticed in these extreme 
cases. May involve several stages or mechanisms.) 

CYCLE (An apparent pattern in which certain events 
or conditions seem to be repeated at regular 
intervals or periods.) 

ENERGY-MATTER (Mutually convertible equivalents of 
interchangeable manifestations of substance which 
enable something to be moved or changed.) 

ENTROPY (An expression of the randomness, disorder 
or chaos in a collection of things, which always 
increases in a closed system.) 

EQUILIBRIUM (A state of "balance or equality" in 
which changes or tendencies to change occur in 
opposite directions, exist or happen at equal 
rates, or are of the same magnitude.) 

EVOLUTION (A series of slow changes that can be 
used to explain how something became the way it 
is or predict what it might become in the future. 
Generally regarded as going from simple to complex.) 

FIELD (A region or space in which something 
influences or affects something else, often without 
direct physical contact.) 

FORCE (A push or pull against resistance.) 

FUNDAMENTAL ENTITIES (Units of structure and 
function useful for explaining certain phenomena.) 

GRADIENT (A situation in which the intensity 
of something varies in a more or less regular • 
pattern.) 

INTERACTION (A situation in which two or more 
things influence or affect each other.) 
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INVARIANCE (A characlcrislic of an object of 
situation which remains constant even though 
other characteristics may change.) 

MODEL (A more or less tentative scheme or structure 
which seems to correspond to a real structure, event 
or class of events that are not directly observable.) 

ORDER (The belief that there is either order in 
nature or that humans arc able to impose order on 
nature. Includes the various schemes or patterns 
used to express this order, e.g., the periodic table 
of elements.) 

ORGANISM (An open dynamic system characterized by 
the processes of life. May be used as a rcpresentalion 
to explain certain nonlife events or thi igs.) 

PERCEPTION (The interaction between the human mind 
and the external world; the mind's interpretation of 
sensory input.) 

POPULATION (A group of fundamental entities having 
similarities or common characteristics.) 

PROBABILITY (The relative certainty, or lack of 
certainty that can be assigned to certain events 
happening in a specified time interval or sequence 
of other events; a number expressing the likelihood 
that a situation or event will happen.) 

QUANTIFICATION (An expression containing a numerical 
component resulting from measurement of some real 
or abstract thing, situation or event.) 

REPLICATION (A belief that doing the same things 
will produce the same results if other conditions 
are the same; a necessary characteristic of scientific 
experiments.) 

RESONANCE (An action within one system which causes 
a similar action within another system.) 

SCALE (A consideration that other characteristics 
or relationships within a system may or may not 
change as its dimensions are mcreased or decreased.) 

SIGNIFICANCE (The belief that certain differences 
that exist or that follow certain actions exceed 
those that would be expected to be caused by chance 
alone.) 
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SYMMETRY (The belief that most, if not all, patterns 
in nature are structurally or functionally balanced, 
or independent of direction.) 



SYSTEM (A group of things or events that can be 
defined, at least in part, by boundaries which enable 
it to be discussed and studied more effectively; 
a set of parts that function 'Ogether as a whole.) 

THEORY (A connected and internally consistent 
group of sentences, equations, models, or a 
combination of these, which serves to explain a 
relatively large and diverse group of things or events.) 

TIME-SPACE (A dimension of the real world which 
separates things and events; time vs. movement.) 

VALIDATION (A belief that similar results obtained 
by two or more different methods reflect an accurate 
representation of the situation being investigated.) 



SCIENTIFIC PROCESSES 



Ci^ASSIFYING (Systematically imposing order on data.) 

COMMUNICATING (Exchanging information.) 

CONTROLLING VARIABLES (Identifying and managing 
factors that may influence an experiment.) 

DEFINING OPERATIONALLY (Defining something by 
describing how it works.) 

DESIGNING EXPERIMENTS (Planning data-gathering 
operations to test hypotheses or answer questions.) 

FORMULATING MODELS (Devising representations that 
explain real things.) 

HYPOTHESIZING (Tentatively accepting an explanation 
as the basis for further investigation.) 

INFERRING (Concluding from known facts or evidence.) 

INTERPRETING DATA (Findi ng patterns or meanings not 
immediately apparent.) 

MEASURING (Using instruments to determine value.) 

OBSERVING (Using the senses to obtain information.) 

PREDICTING (Foretelling from previous information.) 

QUESTIONING (Raising uncertainty.) 

RELATING TIME/SPACE (Describing how duration 
affects distance and vice versa.) 

USING NUMBERS (Expressing ideas, observations and 
relationships in figures, often as complements to 
using words.) 
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APPENDIX C 

CRITERIA FOR 
CONSTRUCTING TESTS WITHOUT RACE/ETHNIC BIAS 

1. Eliminate items that call for backgound experience or information which, though 
common to majority-culture students, may be unknown to students with culturally 
or linguistically different backgrounds. Be sure that all information needed to 

get an item right is given in the item, except the learning for which the item 
tests. 

2. Eliminate offensive stereotypic references to minorities. 

3. Eliminate color symbolism (e.g., yellow-bellied). 

4. Depict human situations which are common to most people. 

5. Use names not strongly associated with a particular ethnic group, OR use names 
from several ethnic groups, evenly distributed. 

6. Avoid references to legendary or historical figures that may be seen as ethnic 
"archetypes." 

7. If real situations or events are described: 

Balance selections of content so that non-majority cultural events and 
perspectives arc adequately represented within a test or a set of items. 

Avoid offensive contrasts in the tone of majority vs. minority-subject ilcnis (e.g., 
portraying Third World or minority culture events as "problems" where majority- 
culture events are portrayed as "achievements"). 



CRITERIA FOR 

CONSTRUCTING TESTS WITHOUT GENDER/SEX-ROLE BIAS 

1. Eliminate offensive stereotypic references to gender groups. 

2. Eliminate "generic" use of male terms (he. man. etc.). 

3. In each test, half the references to humans should be to males and half to females. 

4. One-half of the references to females should depict them in non-stcrcotypic or 
neutral roles. 

5. One-half the references to males should depict them in non-stercotypic or neutral 
roles. 

6. One-half of the titles referring to females should not reflect marital status. 
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APPENDIX D 



WRITING GOOD TEST ITEMS-HOW? 



1. Carefully define the goal, subgoal, or objective to be tested and the purpose of 
testing. 

2. Specify the item characteristics needed (e.g., the vocabulary, signs/symbols, etc. 
to be used or excluded for the students, subjects, and purposes involved). 

3. Work in small groups to write items for the same goals or objectives and testing 
purposes. Begin with discussion; spend some time with resource materials (texts, 
etc.); then write for half an hour to 45 minutes. Stop and share items, noting 
questions and problems. Agree on improvements and put aside items that the 
group has trouble "fixing" to everyone's satisfaction. Repeat the process. 

Before the next session, hvvt all items typed for review and editing. 

4. Wait a few days to a week before reviewing items. Writers should review and 
edit their items and those written by others in the group looking for confusing 
text or responses, or other problems. Annotate typed drafts of items for an 
editor to revise, or revise together and annotate for retyping. 

5. Have another group review retyped items for race/ethnic and sex-role bias. 

6. Have original writers or other qualified reviewer(s) (e.g., an editor, a subject 
matter specialist, another teacher group) give the items a final check. Decide 
on the final forms of items and discard or put aside those with unresolved 
problems. 

7. Try out items with students who have been instructed on the 
goals/subgoals/objcctivcs you*re measuring. 

8. Use appropriate statistical and/or other validation procedures to determine 
which items "work": that is, which items the students with the learning get 
right and the students without it get wrong. 
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APPENDIX E 



CHARACTERISTICS OF 
A HIGH-QUALITY MULTIPLE CHOICE TEST ITEM 



1. Item is logically content-valid. (It measure the goal to which it is related, or it 
can be used to measure an identifiable goal.) 

2. Clear, appropriate, simplest possible text, with terms used accurately. 

3. Freedom from racial, ethnic or sex-roic bias or stereotyping. 

4. No extraneous information; no tricks or traps. 

5. No give-aways: answer choices about the same length, no grammatical cues, etc. 

6. No use of double negatives. 

7. One right or best answer, with plausible distractors. (If the "best" answer is 
called for, criteria for "best" arc included.) 

8. Logical ordering of answer choices. 

9. All information is provided to choose the con ct answer (except that for which 
the item tests) if the student does have the knowledge or skill the item is written 
to test. 

10. Vocabulary used in the item is standard usage, spelled and punctuated correctly, 
and appropriate to grade level and subject. 

11. Graphs or illustrations are clear and accurate, or could be made so. 

12. Judicious (or no) use of humor or whimsey. 
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MEMBERSHIP FORM 
1987-88 Membership Year 

IMoasc pnwulc the information requested below and mail to: Susan Smoycr, NWRA Seience l^rojccl, 700 Priiigle 
Parkway S.E.. Salem OR 97310. 

DISTRICT/AGENCY 

ADDRESS.^ 

CITY .STATE ZIP 

CONTACT name; ^TELEPHONE ( ) 

POSITION 

□ New Member ($2,000) □ Renewal ($1,000) 

D Enclosed is a check (payable to NWEA Science Project) in the amount of $ for 1987-88 

membership fees. 

D Enclosed is a Purchase Order in the amount of $ for 1987-88 membership fees. 



If you want others in your district or agency to receive science project mailings, please provide the following 
information: 

NAME/POSITION ADDRESS PHONE 

1. 



2. 



3. 



Please list two others in your agency to whom NWEA mailings should be directed: 

NAME/POSITION ADDRESS PHONE 

1. 
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ORDER FORM 



Science Curriculum Surveys 

D K-6 n Chemistry 

LJ Middle School/Junior High Q Physics 

D Biology n Earlh/Space 

\3 Survey Scoring/Reporting Service^ 



dl Curriculum Planning Materials 

D Science Test Item Collection 
(approximately 7,500 items 
on card stock) 

Q Electronic version of 
the Science test items^ 

Q Focus on Science Concepts 

(Set includes Cycle, Equilibrium, 
Force, Model and Perception; 
5 copies of each concept.) 

C High School Science Programs 
Managing for Excellence 

n The Study of Learning Environments 



MEMBERS 
Included in membership fee 



One-time processing for "o to 50 
surveys included in membership 
fee. $3.50 per survey over 50. 

Included in membership fee 

Included in membership fee 



Included in membership fee 



All publications included in 
membership fee. Additional 
copies at non-member price. 



NON^MEMBERS 

S20 per survey 
$100 set of six 



$^.00 per sur\ey 
(with $100 
minimum) 

$150 

$2,000 



$2,000 



$45 



$6 



NAME- 



DISTRICT/AGENCY^ 



ADDRES5. 



CITY, 



TELEPHONE ( 



STATE 



_ZIP_- 



Please enclose check (payable to NWEA Science Project) or purchase order and mail to Susan Smoycr. NWEA 
Science Project. 700 Pringle Parkway S.E., Salem. OR 97310. 



^ Please indicate if you intend to purchase the project's scoring and reporting service, but do not include 
this amount in your check or purchase order. You will be billed when your surveys are processed. 

2 Available October, 1988. 
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